{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BWV9n-Gi3iL_"
      },
      "source": [
        "# About This Demo\n",
        "\n",
        "The GPU transcoding and filter module demo shows:\n",
        "1. Common video/image filters in BMF accelerated by GPU\n",
        "2. How to write GPU modules in BMF\n",
        "\n",
        "The demo builds a transcoding pipeline which fully runs on GPU:\n",
        "\n",
        "decode->scale->flip->rotate->crop->blur->encode\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YIZ68QEsA7xW"
      },
      "source": [
        "\n",
        "# Get Started\n",
        "\n",
        "## 1. Environment Setup\n",
        "*   install the BMF\n",
        "*   make sure the GPU environment is ready"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZfCJBbnSBAR6"
      },
      "source": [
        "### 1-1 pip install BMF packages"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SGS2FIiylx2I"
      },
      "source": [
        "To install a GPU supported version of BMF:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IjiiVnD9Aua-"
      },
      "outputs": [],
      "source": [
        "!pip uninstall -y BabitMF-GPU\n",
        "!pip3 install BabitMF-GPU"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Uf9TnSHx_C0a"
      },
      "outputs": [],
      "source": [
        "%env LIBRARY_PATH=$LIBRARY_PATH:/usr/local/lib/python3.10/dist-packages/bmf/lib"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KVQKIPamWnPA"
      },
      "source": [
        "### 1-2 verify the FFmpeg libraries is installed and version is correct\n",
        "\n",
        "Part of feature in BMF framework utilizes the FFmpeg demuxer/muxer/codec and filter as the built-in modules for video processing. **If the capability of ffmpeg is needed,it's neccessary for users to install supported FFmpeg libraries before using BMF.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WiPmTkhfaXft"
      },
      "source": [
        "**Optional step**\n",
        "\n",
        "Install ffmpeg and related libraries. For this demo, we don't have to do this step, because ffmpeg libraries are already installed in the Google Colab environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ARXN39-VaQlg"
      },
      "outputs": [],
      "source": [
        "!sudo apt install ffmpeg"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PcLbXWWSYMfQ"
      },
      "source": [
        "List the ffmpeg libraries. It is expected that the related libraries such libavcodec, libavformat are installed. The output should be showned as below:\n",
        "![image.png]()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_ZcqxtWJnvea"
      },
      "outputs": [],
      "source": [
        "!sudo apt install libdw1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GLnRGvl5W00c"
      },
      "outputs": [],
      "source": [
        "!dpkg -l | grep -i ffmpeg"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bcLVtZ55YeWB"
      },
      "source": [
        "Display the FFmpeg version. **BMF supports the FFmpeg verions above 4.2.x.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "27HbW50PZLRG"
      },
      "outputs": [],
      "source": [
        "!ffmpeg -version"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DU1NJhoXAImJ"
      },
      "source": [
        "### 1-3 install CV-CUDA\n",
        "\n",
        "The GPU modules are implemented using CV-CUDA, **it's necessary to install CV-CUDA if you want to run the modules.**\n",
        "\n",
        "To install CV-CUDA, we need to collect the pre-built binaries from github then install them using apt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yRg2EMZzB1Lb"
      },
      "outputs": [],
      "source": [
        "!wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.3.1-beta/nvcv-lib-0.3.1_beta-cuda12-x86_64-linux.deb\n",
        "!wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.3.1-beta/nvcv-dev-0.3.1_beta-cuda12-x86_64-linux.deb\n",
        "!wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.3.1-beta/nvcv-python3.10-0.3.1_beta-cuda12-x86_64-linux.deb\n",
        "!wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.3.1-beta/nvcv_python-0.3.1_beta-cp310-cp310-linux_x86_64.whl\n",
        "\n",
        "!sudo apt install ./nvcv-lib-0.3.1_beta-cuda12-x86_64-linux.deb ./nvcv-dev-0.3.1_beta-cuda12-x86_64-linux.deb ./nvcv-python3.10-0.3.1_beta-cuda12-x86_64-linux.deb"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "loMsWzrAD8Iy"
      },
      "source": [
        "Verify that CV-CUDA has been installed properly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YFW8_8VoECnD"
      },
      "outputs": [],
      "source": [
        "import cvcuda\n",
        "print(cvcuda.__version__)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AuU2xFfsb3VV"
      },
      "source": [
        "### 1-3 install wurlitzer\n",
        "\n",
        "This package is installed to show the BMF C++ logs in the colab console, otherwise only python logs are printed. This step is not necessary if you're not in a Colab or iPython notebook environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VoXqpHLh5aBM"
      },
      "outputs": [],
      "source": [
        "!pip install wurlitzer\n",
        "%load_ext wurlitzer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hC4ZKWhOd7g6"
      },
      "source": [
        "## 2. BMF GPU Transcoding & Filter Demo\n",
        " Now let's set up a gpu transcoding pipeline with common filters. The pipeline will be complete run on GPU, which means the data does not need to be copied back to CPU. We should always avoid CPU-GPU data movement as much as possible, this is an important practice in terms of performance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Q-x00orgL-S"
      },
      "source": [
        "### 2-1 Fetch code from github\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FZKn0pB1p1CI"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/BabitMF/bmf.git\n",
        "%cd bmf/bmf/demo/gpu_module"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2-1 Fetch test video from bmf repo\n",
        "\n"
      ],
      "metadata": {
        "id": "tTv-rR1tSz-j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://github.com/BabitMF/bmf/releases/download/files/files.tar.gz\n",
        "!tar -zvxf files.tar.gz\n",
        "!cp files/big_bunny_10s_30fps.mp4 ."
      ],
      "metadata": {
        "id": "WEwZYOiHS5RY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PYkDR2wj_oZz"
      },
      "source": [
        "### 2-2 Run the demo\n",
        "\n",
        "The example GPU modules are implemented in the following files:\n",
        "\n",
        "*   scale_gpu.py\n",
        "*   flip_gpu.py\n",
        "*   rotate_gpu.py\n",
        "*   crop_gpu.py\n",
        "*   blur_gpu.py\n",
        "\n",
        "You can tell what the module does by its name. Please refer to `bmf/bmf/docs/example/Example_GpuModule.md` for detailed documentation of the GPU modules. Now let's run these modules using the following Python script\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P_gQVJhF_rTx"
      },
      "outputs": [],
      "source": [
        "import bmf\n",
        "\n",
        "def test():\n",
        "    input_video_path = \"./big_bunny_10s_30fps.mp4\"\n",
        "    output_path = \"./output.mp4\"\n",
        "\n",
        "    graph = bmf.graph()\n",
        "    video = graph.decode({\n",
        "        \"input_path\": input_video_path,\n",
        "        \"video_params\": {\n",
        "            \"hwaccel\": \"cuda\"\n",
        "        }\n",
        "    })\n",
        "    (\n",
        "        video['video']\n",
        "            .module('scale_gpu', {\"size\": '1920x1080', 'algo': 'cubic'})\n",
        "            .module('crop_gpu', {'x': 960, 'y': 540, 'width': 480, 'height': 480})\n",
        "            .module('flip_gpu', {'direction': 'h'}) # direction can be 'vertical', 'h', 'horizontal' or 'both'\n",
        "            .module('rotate_gpu', {'angle': 'pi/8'})\n",
        "            .module('blur_gpu', {'op': 'gblur', 'sigma': [0.7, 0.7], 'size': [5, 5]})\n",
        "            .encode(None, {\n",
        "                \"output_path\": output_path,\n",
        "                \"video_params\": {\n",
        "                    \"codec\": \"hevc_nvenc\",\n",
        "                    \"pix_fmt\": \"cuda\",\n",
        "                }\n",
        "            })\n",
        "            .run()\n",
        "    )\n",
        "\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    test()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iYY1yu8zBubm"
      },
      "source": [
        "Display the input and output video streams:"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import HTML\n",
        "from base64 import b64encode\n",
        "\n",
        "def show_video(video_path, video_width = 800):\n",
        "\n",
        "  video_file = open(video_path, \"r+b\").read()\n",
        "\n",
        "  video_url = f\"data:video/mp4;base64,{b64encode(video_file).decode()}\"\n",
        "\n",
        "  return HTML(f\"\"\"<video width={video_width} controls><source src=\"{video_url}\"></video>\"\"\")"
      ],
      "metadata": {
        "id": "Dfj-rquYIarr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# show output video\n",
        "show_video(\"./big_bunny_10s_30fps.mp4\")"
      ],
      "metadata": {
        "id": "gbqmDKQsIjhz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6QHnqMjlBz7P"
      },
      "outputs": [],
      "source": [
        "# show output video\n",
        "show_video(\"./output.mp4\")"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
